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Abstract 

Inference of online social network users’ attributes 
and interests has been an active research topic. Ac¬ 
curate identification of users’ attributes and inter¬ 
ests is crucial for improving the performance of 
personalization and recommender systems. Most 
of the existing works have focused on textual con¬ 
tent generated by the users and have successfully 
used it for predicting users’ interests and other 
identifying attributes. However, little attention has 
been paid to user generated visual content (images) 
that is becoming increasingly popular and perva¬ 
sive in recent times. We posit that images posted 
by users on online social networks are a reflection 
of topics they are interested in and propose an ap¬ 
proach to infer user attributes from images posted 
by them. We analyze the content of individual im¬ 
ages and then aggregate the image-level knowledge 
to infer user-level interest distribution. We employ 
image-level similarity to propagate the label infor¬ 
mation between images, as well as utilize the im¬ 
age category information derived from the user cre¬ 
ated organization structure to further propagate the 
category-level knowledge for all images. A real 
life social network dataset created from Pinterest 
is used for evaluation and the experimental results 
demonstrate the effectiveness of our proposed ap¬ 
proach. 

1 Introduction 

Online Social Networks (OSNs) such as Facebook, Twitter, 
Pinterest, Instagram, etc. have become a part and parcel 
of modern lifestyle. A study by Pew Research centr^ re¬ 
veals that three out of every four adult internet users use 
at least one social networking site. Such large scale adop¬ 
tion of OSNs and active participation of users have led to 
research efforts studying relationship between users’ digital 
behavior and their demographic attributes (such as age, in¬ 
terests, and preferences) that are of particular interest to so¬ 
cial science, psychology, and marketing. A large scale study 

^ http://www.pewinternet.org/2013/12/30/social-media-update- 
2013/ 




Figure 1: Example pinboards from one typical Pinterest user. 


of about 58,000 Facebook users performed by Kosinski et 
al. |2013| reveals that digital records of human activity can 
be used to accurately predict a range of personal attributes 
such as age, gender, sexual orientation, political orientation, 
etc. Likewise, there have been numerous works that study 
variations in lang uage used in social media with age, gender, 
personality, etc. I Burger et al., 2011 Bamman et al., 2014 


Schwartz g/ al., 201 3| . While most of the popular OSNs 

studied in literature are mostly text based, some of them 
(e.g., Facebook, Twitter) also allow people to post images and 
videos. Recently, OSNs such as Instagram and Pinterest that 
are majorly image based have gained popularity with almost 
20 billion photos already been shared on Inst^ram and an 
average of 60 million photos being shared dail}0 

The most appealing aspect of image based OSNs is that vi¬ 
sual content is universal in nature and thus, not restricted by 
the barriers of language. Users from different cultural back¬ 
grounds, nationalites, and speaking different languages can 
easily use the same visual language to express their feelings. 
Hence, analyzing the content of user posted images is an ap¬ 
pealing idea with diverse applications. Some recent research 
efforts also provide support for the hypothesis that images 


^ http ://instagram. com/pres s/ 


















posted by users on OSNs may prove to be useful for learn¬ 
ing va rious p ersonal and social attributes of users. Lovato 
et al. | 2013a| proposed a method to learn users’ latent vi¬ 
sual preferences by extracting aesthetics and visual features 
from images favorited by users on Flickr. The learned mod¬ 
els can be used to predict images likely to be favorited by the 
user on Flickr with reasonable accuracy. Cristani et al. (20T^ 
infer personalities of users by extracting visual patterns and 
features fr om im ages marked as favorites by users on Flickr. 
Can et al. |2013| utilize the visual cues of tweeted images in 
addition to textual and structure-based features to predict the 
retweet count of the posted image. Motivated by these works, 
we investigate if the images posted by users on online social 
networks can be used to predict their fine-grained interests or 
preferences about different topics. To understand this better, 
Figure shows several randomly selected pinboards (collec¬ 
tion of images, as they are called in Pinterest) for a typical 
Pinterest user as an example. We observe that different pins 
(a pin corresponds to an image in Pinterest) and pinboards 
are indicative of the user’s interests in different topics such as 
sports, art and food. We posit that the visual content of the 
images posted by a user in an OSN is a reflection of her in¬ 
terests and preferences. Therefore, an analysis of such posted 
images can be used to create an interest profile of the user 
by analyzing the content of individual images posted by the 
user and then aggregating the image-level knowledge to infer 
user-level preference distribution at a fine-grained level. 


1.1 Problem Formulation and Overview of 
Proposed Approach 

Problem Statement: Given a set X of images posted by the 
user u on an OSN, and a set C of interest categories, out¬ 
put a probability distribution over categories in C as the in¬ 
terest distribution for the user. In order to solve this prob¬ 
lem, we first need to understand the relationships between 
different categories (topics) and underlying characteristics (or 
features) of user posted images in the training phase. These 
learned relationships can then be used to predict distribution 
over different interest categories by analyzing images posted 
by a new user. Even though state-of-the-art machine learning 
algorithms, especially developments in deep learning, have 
achie ved significant results fo r individual image classifica¬ 
tion |Krizhevsky et al, 2012| , we believe that incorporat¬ 
ing OSN and user specific information can provide further 
performance gains. Different image based OSNs offer users 
capability to group together similar images in the form of al¬ 
bums/pinboards, etc. In Pinterest, users create pinboards for a 
given topic and collect similar images in the pinboard. Given 
this human created group information, it is reasonable to as¬ 
sume that strong correlations exist between objects belonging 
to the same curated group or categorization. For example, a 
user may have two pinboards belonging to the Sports cate¬ 
gory, one for images related to soccer and one for images 
related to basketball. Therefore, even though all images in 
the two pinboards will share some common characteristics, 
images within each pinboard will share some additional sim¬ 
ilarities. Motivated by these observations, we employ image 
level and group level label propagation in order to build more 
accurate learning models. We employ image-level similar¬ 


ity to propagate the label information between images and 
category correlations are employed to further propagate the 
category-level knowledge for all images. 


2 Related Work 


2.1 User Profiling from Online Social Networks 

It is discovered that in social network people’s relationship 
follows the rule birds of a feather flock together |McPhers^ 
et al, 2Q()T| . Similarly, people in online social network also 
exhibit such kind of patterns. Online social network users 
connected with other users may be due to very different rea¬ 
sons. For instance, they may try to rebuild their real world 
social networks on the online social network. However, most 
of the time, people hope to show part of themselves to the 
rest of the world. In this case, the content generated by online 
social network users may help us to infer their characteris¬ 
tics. Therefore, we are able to build more accurate and more 
targeted systems for prediction and recommendation. 

There are many related works on profiling different on¬ 
line users’ attributes. Location is quite important for adver¬ 
tisement targeting, local customer personalization and m any 
novel location based applications. In Cheng et al. (Mo) , the 
authors proposed an algorithm to estimate a city le vel location 
for Twitter users. More recently, Li et al. |2012| proposed a 
unified discriminative influence model to estimate the home 
location of online social users. They unified both the social 
network and user-centric signals into a probabilistic frame¬ 
work. In this way, they are able to more accurately estimate 
the home locations. Meanwhile, since most social network 
users tend to have multiple social network accounts, where 
they e xhibit different behaviors on these platforms. Reza and 
Huan |2013| proposed MOBIUS for finding a mapping of so¬ 
cial network users across different social platforms accord¬ 
ing to_people’s behavioral patterns. The work in Mislove et 
al. |2010| also proposed an approach trying to inferring user 
profiles by employin g the social network graph. In a recent 
work, Kosinski et al. |2013 1 revealed the power of using Face- 
book to predict private traits and attributes for Facebook vol¬ 
unteer users. Their results indicated that simple human social 
network behaviors are able to predict a wide range of human 
attributes, including sexual orientation, ethnic origin, politi- 
cal views, religion, intelligence and so on. Li et al. l [2014| 
defined discriminative correlation between attributes and so¬ 
cial connections, where they tried to infer different attributes 
from different circles of relations. 


2.2 Visual Content Analysis 

Visual content becomes increasingly popular in all online 
social networks. Recent research works have indicated the 
possibility of using online user generated visual content to 
learn personal attributes. In Kosinsky et al. |2Q13[ , a total of 
58,000 volunteers provided their Facebook likes as well as 
detailed demographic profiles. Their results suggest that dig¬ 
ital records of human activities can be used to accurately pre¬ 
dict a range of personal attributes such as age, gender, sexual 
orientation, a nd politi cal orientation. Meanwhile, the work in 
Lovato et al. |2013b| tried to build the connection between 


































cognitive effects and the consumption of multimedia con¬ 
tent. Image features, including both aesthetics and content, 
are employed to predict online users’ personality traits. The 
findings may suggest new opportunities for both multimedia 
tec hnolog ies and cognitive science. More recently, Lovato et 
al. 1 20141 proposed to learn users’ biometrics from their col¬ 
lections of favorite images. Various perceptual and content 
visual features are proven to be effective in terms of p redict- 
ing users’ preference of images. You, Bhatia and Luo | 2014| 
exploited visual features to determine the gender of online 
users from their posted images. 


3 Proposed Approach 

As discussed in Section the problem of inferring user in¬ 
terests can be considered as an image classification prob¬ 
lem. However, in contrast to the classical image classification 
problem where the objective is to maximize classification per¬ 
formance at individual level, we are focused more on learning 
the overall user-level image category distribution, which in 
turn yields users’ interests distribution. In our work, we use 
data crawled from pinterest.com, which is one of the most 
popular image based social networks. In Pinterest, users can 
share/save images that are known as pins. Users can catego¬ 
rize these pins into different pinboards such that a pinboard is 
collection of related pins (images). Also, while creating a pin¬ 
board the user has to chose a descriptive category label for the 
pinboard from a pre-defined list specified by Pinterest. There 
are a total of 34 available categories for users to chose from 
(listed in Table [^. A typical Pinterest manages/owns many 
different pinboards belonging to different categories and each 
pinboard will contain closely related images of the same cat¬ 
egory. Pinterest users mainly use pinboards to attract other 
users and also to organize the images of interest for them¬ 
selves. Therefore, often times they will choose interesting 
and high-quality pins to add to their pinboards and one can 
use these carefully selected and well organized high quality 
images to infer the interests of the user. 


3.1 Training Image-level Classification Model 

We employ the Convolutional Neural Network (CNN) to train 
our image classifier. The main architecture comes from the 
Co nvoluti onal Neural Network proposed by Krizhevsky et 
al. 1 ^12] , that has achieved state of the art performance in 
the challenging ImageNet classification problem. We tra in 
our model on top of the model of Krizhevsky et al. |M^ ). 
For a detailed description of the architecture of the model, we 
direct the reader to the original paper. In particular, we fine- 
tune our model by employing images and labels from Pinter¬ 
est. For each image, we assign the label of the image to be 
the same as the label of the pinboard it belongs to. More de¬ 
tails on the preparation of training data for our model will be 
discussed in the experimental section. From the trained deep 
CNN model, it is possible to extract deep features for each im¬ 
age. These features, also known as high-level features, have 
shown to be more appropriate for performing various other 
image related ta sks, such as im age content analysis and se¬ 
mantic analysis jBengio, 2009| . Specifically, we extract the 
deep CNN features from the last fully connected layer of our 


trained CNN model and employ these deep features for label 
propagation as described in the next section. It is notewor¬ 
thy that this work differs from image annotation, which deals 
with individual images and have been extensively studied. As 
will become more clear, the approach we take in the form of 
label propagation, is designed to exploit the strong connection 
between the images curated within and across collections by 
users. For the same reason, the typical noise in user data is 
suppressed in Pinterest due the same user curation process. 

3.2 Image and Group Level Label Propagation for 
Prediction 

During the prediction stage, we try to solve a more general 
problem, where we are given a collection of images without 
group information. However, we want to predict the users’ 
interests from these unorganized collection of images. We 
propose to u se labe l propagation for image labels. The work 
in Yu et al. | |20lH ) also tried to use collection information 
for label propagation. However, differently from their work, 
where collection information is employed to extract sparsity 
constraint to the original label propagation algorithm, we pro¬ 
pose to impose the additional group-level similarity to further 
propagate the image labels for the same user. 

Meanwhile, we observe that for most of the categories 
in Table T] the average distance between images in the same 
pinboard have closer distance than images in the same cate¬ 
gory. Figure shows the average distance between images 
in the same pinboards and the average distance between im¬ 
ages in different pinboards but in the same categories. The 
distance is represented by the squared Euclidean distance be¬ 
tween deep features from last layer of Convolutional Neural 
Network. Intuitively, this can be explained by the fact that 
most of the categories are quite diverse in that they can con¬ 
tain quite different sub-categories. On the other hand, users 
generally create pinboards to collect similar images into the 
same group. These images are more similar in that they are 
more likely to be in the same sub-category of the category 
label chosen by the user. Hence, the motivation for imple¬ 
menting label propagation at user level. 

Let n be the number of categories, we define matrix G G 
p^nxn affinity matrix between these n categories 

(G is normalized such that the columns of G sum 1). The 
intuitive idea is that in general, there exists some correlation 
between a person’s interest. If one user likes sports, then he is 
likely to be interested in Health & Fitness. This kind of infor¬ 
mation may help us to predict users’ interest more accurately, 
especially for users with very few images in their potential in¬ 
terest categories. Therefore, during the label propagation, we 
also consider the propagation of category-level information. 

To incorporate the group level information into our model, 
we define the following iterative process for image label prop¬ 
agation. 

= (1 - K)WY^G + (1) 

where is the initial prediction of image labels from the 
trained CNN model, W is the normalized similarity matrix 
between images and A is a diagonal matrix. Following Yu et 



















Table 1: List of 34 categories in Pinterest. 


Animals 

Architecture 

Art 

Cars & Motorcycles 

Celebrities 

Design 

DIY & Crafts 

Education 

Film, Music & Books 

Food & Drink 

Gardening 

Geek 

Hair & Beauty 

Health & Fitness 

History 

Holidays & Events 

Home Decor 

Humor 

Illustrations & Posters 

Kids 

Men’s Fashion 

Outdoors 

Photography 

Products 

Quotes 

Science & Nature 

Sports 

Tattoos 

Technology 

Travel 

Weddings 

Women’s Fashion 

Other 





Figure 2: Average distance between images in different groups. 


al. | |201l) ), A is defined as 

yo. 

= max . (2) 

We can consider the above process as two stages. In the first 
stage (1 — K)WY^, we use the similarity between the images 
to propagate the labels at image level. In the second stage, 
the group relationship matrix G is employed to further prop¬ 
agate the group relationship to all images, i.e. we multiply 
matrix G with the results of (1 — A)WY^. Next, we present 
the convergence analysis of the proposed label propagation 
framework. 

Convergence Analysis 

From Eqn. we have the following formula for 

t 

y^+i = ((1 _ A) v|y)^+i ((i _ A) w)^ AY^G^ 

i=0 

(3) 

Since we have 0 < Xij^Wij^Gij < 1 for all i, j, therefore 
((1 - A) = 0. It follows that 

t-i 

lim = y ((1 - A) wy AY^G\ (4) 

t^oc ^^ 

i=0 

According to the theo rem that the produ ct of two converging 
sequences converges | |Rosenlicht, 19681 , we define two dif¬ 
ferent matrix series = ((1 — A ) W) AF^ and = G^. 
For matrix series it converges |Yu et al, 201 1| . For ma¬ 
trix series it also converges since p{B) < 1. There¬ 
fore, we conclude that the product of the two matrix se¬ 
ries A^ and B^ also converges. More importantly, we have 

^In our implementation, we use Jaccard index to calculate G, see 
the experimental section for detailed description. 


Algorithm 1 User Profiling by Group Constraint Label Prop¬ 
agation 

Require: X = {xi, X 2 ,..., xat} a collection of images 
M: Fine-tuned Imagenet CNN model 
G: Correlation matrix between the labels 

1: Predict the categories Y^ G (K is the number of 

categories) of X using the trained CNN model M 

2: Extract deep features from M for all X. 

3: Calculate the similarity matrix W' between X using 

Gaussian kernel function W'{i,j) = exp(— ), 

where Xi and Xj are the deep features for image i and j 
respectively. 

4: Normalize W' to get W = where D is diagonal 

matrix with Da = W'j . 

5: Calculate the diagonal matrix A according to Eqn.([^. 

6: Calculate the affinity matrix between G between different 
categorie^ 

7: Initialize, iteration index t = 0 

8 : repeat 

9: Employ Eqn.([^ to update according to Y^ 

10: until Convergence or t reaches the maximum iteration 
number 

11: Normalize rows of Y^ G to get Y'^ G R^^^. 

12 : return Y'^. 


Y,i Ai = {I - A) AF^ and Y^iBi = {I -Q) ^ All the ele¬ 
ments of Ai and Bi are non-negative, which leads to the fact 
that 5];. AiBi <p Ai){^. Bi). We use <p to represent 
the element-wise comparison between two matrix. In other 
words, we have a upper bound for F^, together with the fact 
that AiBi > 0, we conclude that F^ converges. 

Meanwhile, since we implement label propagation in each 
user’s collection of images, thus the main computational and 
storage complexity is in the order of 0(nf), where rii is the 
number of images for user i. 



























































Algorithmic summarizes the main steps using the proposed 
group constrained label propagation framework. Note that in 
step|C we use a Gaussian kernel function to calculate the sim¬ 
ilarity between two images using the deep features. However, 
one can employ other techniques such as lo cally linear em¬ 
bedding (LLE) i Donoho and Grimes, 2003 1 to calculate the 
similarity between different instances. 

Eventually, we obtain users’ interests distribution by aggre¬ 
gating the label distribution of the collections of their images. 
In other words, we simply sum up the label distribution of 
individual images and then normalize it to produce the users’ 
interest prediction results. 


4 Experiments and Evaluations 

To evaluate the proposed algorithm, we crawl data from Pin- 
terest according to a randomly chosen user list consisting of 
748 users. Tablegives the statistics of our crawled dataset. 
This dataset is used to train our own deep convolutional neu¬ 
ral network. We use 80% of these images as the training data 
and the remaining 20% as validation data. We also down¬ 
loaded another 77 users’ data as our testing data. Since each 
pinboard has one category, we use the category label as the 
label for all the images in that pinboard. 


Table 2: Statistics of our dataset. 



Training and Validating 

Testing 

Num of Users 

748 

77 

Num of Pinboards 

30,213 

1,126 

Num of Pins 

1,586,947 

66,050 


We train th e CNN on top of the Im ageNet convolutional 
neural model | ]Krizhevsky et ah, 2012|. W e use the publicly 
available implementation Caffe l |Jia, 2()T3| to train our model. 
All of our experiments are evaluated on a Linux X86_64 ma¬ 
chine with 32G RAM and two NVIDIA GTX Titan GPUs. 
We finish the training of CNN after a total of 200, 000 itera¬ 
tions. 

In the proposed group constrained label propagation, we 
need to know the similarity matrix G between all categories. 
We propose to learn the similarity matrix G from pinterest 
users’ behaviors of building their collections of pinboards. 
No additional textual semantic or visual semantic informa¬ 
tion is employed to build the similarity matrix. In our exper¬ 
iments, we employ Jaccard index to calculate the similarity 
coefficient between different categories in Table Specifi¬ 
cally, the entry of Gij is defined as the Jaccard index between 
category i and j, which is the ratio between the number of 
users who have pinboards of both categories i and j and the 
number of users who have pinboards of category i or j. Eig- 
ure|^ shows the coefficients between all different categories. 
It shows that users prefer to choose some of the categories 
together, which may suggest potential category recommen¬ 
dations for Pinterest users. 

4.1 Evaluation Criteria 

To evaluate the performance of different models, we employ 
two different criteria. 1) Normalized Discounted Cumula¬ 
tive Gain (NDCG) score NDCG is a popular measure for 
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Eigure 3: Jaccard similarity coefficients between different 
categories in T able 

ranking tasks jCroft et al, 2010[ . Discounted Cumulative 
Gain at position n is defined as 


DGGn 


n 

Pi + x 

i=l 


Pi 

log2(0’ 


(5) 


where piis the relevance score at position i. In our case, we 
use the predicted probability as the relevance score. NDCG 
is the normalized DCG, where the value is between 0 and 
1. To calculate the NDCG, for a user ui, we first get the 
ground truth distribution of his interest according to the cat¬ 
egory labels of his pinboards and then rank the ground truth 
categories in descending order of their distribution probabil¬ 
ities. Note that the pinboard labels of Ui may not contain all 
the categories and our trained classifier may give a class dis¬ 
tribution over all the categories. Therefore, when calculating 
the NDCG score we add a prior to the interest distribution 
of each user. We first calculate the prior category distribu¬ 
tion po from all the training and validating data set according 
to their labels. Then, for each testing user i we first calcu¬ 
late the category distribution of pi according to the ground 
truth labels. Next, we smooth the distribution by updating 
Pi = Pi + 0.1 * Po and then normalize it to get the ground 
truth interest distribution for user i. In this way, we are able to 
calculate the NDCG score for each user according to the pre¬ 
diction of the model. 2) Recall®Rank K In this metric, we 
firstly build the ground truth for one user Ui , we rank all of his 
categories in descending order according to the distribution 
probabilities. Then, we calculate the shared top ranked cate¬ 
gories with the ground truth. Users’ data in Pinterest may be 
incomplete. Eor example, user i may be interested in Sports 
and Women’s Eashion. However, she only has created pin¬ 
boards belonging to Women’s Eashion. She has not created 
Sports related pinboards. However, it does not mean that she 
is not interested in Sports. Thus, it makes sense to evaluate 
different algorithms’ performance using the Recall®Rank k. 
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Figure 4: Filters of the first convolutional layer. 

4.2 Experimental Results 



NDCG Intervals 


Figure 5: Distribution of NDCG scores for three different ap¬ 
proaches. 


We report the results for three different models. First one 
is the results from our fine-tuned CNN model, i.e. the initial 
predicti on for label prop agation. Second model is label prop¬ 
agation ]Yu et al, 201 1[ (LP) and the last one is the proposed 
group constrained label propagation (GLP). 

We first train our CNN model using the data in Table 
Figure shows the learned filters in the first layer of the 
CNN model, which is consistent with t he filters learned in 
other image related deep learning tasks |Zeiler and Fergus, 


2013| . These Gabor-like filters can be e mployed to repli 


cate the properties of human brain V1 area I Lee et al , 2008) , 
which can detect low-level texture features from raw input 
images. Next, we employ this model to predict the labels of 
the testing data, which is going to be the initial label predic¬ 
tion for LP and GLP models. 


Table 3: Mean and Standard deviation of NDCG score 


Model 

Mean 

STD 

CNN 

0.692 

0.179 

LP 

0.818 

0.138 

GLP 

0.826 

0.138 


For both LP and GLP models, we choose the S in the Gaus¬ 
sian kernel to be the variance of dista nce between each pair 
of deep features |Von Luxburg, 200^ . We set the maximum 
number of iterations to be 100 for both model^ Figure 
shows the performance of using NDCG for the three mod¬ 
els. The results shows that both LP and GLP try to move the 
distribution to the right, which suggests that both models can 
improve the performance in terms of NDCG score for most 
of the users. To quantitatively analyze the results, we give 
the mean and the standard derivation of the NDCG scores 
in TableBoth LP and GLP improve the NDCG score from 
about 0.69 to over 0.80 and reduce the standard deviation. 
GLP shows slight advantage over LP in terms of both mean 
and standard deviation of NDCG score. 


^In our experiments, the increase of iteration number over 100 
leads to similar results with the reported results in this work. 



Figure 6: Recall for different K for the three models re¬ 
spectively. 

The evaluation results of using Recall @K are shown 
in Figure Differently from the NDCG score, CNN and 
GLP show better recall performance then LP for all different 
values of Ks. Meanwhile, GLP consistently outperforms the 
CNN model. The results suggest that without the propagation 
of category or group similarity, label propagation may cause 
the increase of irrelevant categories, which leads to poor re¬ 
call. 

Besides the user-level interests prediction, we are also in¬ 
terested in whether or not label propagation can improve 
the performance of image-level classification performance. 
Since we have the pinboard labels as the ground-truth labels 
for each testing image, we report the accuracy of the three 
models on the testing images. Table summarizes the accu¬ 
racy. Indeed, the performance of these three models is quite 
similar. These results may suggest that the label propaga¬ 
tion have limited impact on the overall label distribution of 
individual image, i.e. the max index in the probability distri¬ 
bution of labels. However, they may change the probability 
distribution, which may benefits the overall user-level inter¬ 
ests estimation. 

Table 4: Accuracy on Image-level classification of different 
models. 


Model 

Accuracy 

CNN 

0.431 

LP 

0.434 

GLP 

0.434 


5 Conclusions 

We addressed the problem of user interest distribution by an¬ 
alyzing user generated visual content. We framed the prob¬ 
lem as an image classification problem and trained a CNN 
model on training images spanning over 748 users’ photo al¬ 
bums. Taking advantage of the human intelligence incorpo¬ 
rated through the user curation of the organized visual con¬ 
tent, we used image-level similarity to propagate the label in¬ 
formation between images, as well as utilized the image cat¬ 
egory information derived from the user created organization 
structure to further propagate the category-level knowledge 






























































for all images. Experimental evaluation using data derived 
from Pinterest provided support for the effectiveness of the 
proposed method. In this work, our focus was on using im¬ 
age content for user interest prediction. We plan to extend our 
work to also incorporate user generated textual content in the 
form of comments and user profile information for improved 
performance. 
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